1. Field of the Invention
The present invention relates to a technology for measuring depth information on a subject in an image by using image information.
2. Description of the Related Art
Conventionally, as a method for acquiring the depth of an imaging scene from an image acquired by an image pickup apparatus, a depth from defocus (DFD) method is proposed. In the DFD method, a plurality of images having different blurs are acquired by controlling imaging parameters of an image pickup optical system, and the correlation value of the mutual blurs is calculated on a per pixel basis in the plurality of images. In a case where only one pixel is used in depth calculation, the spread of the blur becomes obscure or the depth calculation becomes unstable due to a noise, and hence the correlation value is calculated with surrounding pixels included in the calculation. Finally, by using the relationship between the blur caused by the image pickup optical system and a subject depth, depth information on the subject is calculated based on the calculated correlation value of the blur. The DFD method has advantages that the depth can be calculated using one image pickup system, and a depth map can be calculated because the DFD method allows the calculation of the depth for each pixel.
However, there has been a problem that a processing time is significantly increased when the number of pixels for depth calculation is increased. In addition, in a case where the depth calculation is performed by using the surrounding region of the pixel for depth calculation, there has been a problem that accuracy is reduced when a plurality of depths are present in the surrounding region used in the depth calculation.
In view of the above problems, in Japanese Patent Application Laid-open No. 2010-117593, the number of redundant depth measurement points is reduced by segmenting an input image into regions and changing the density of the depth measurement point according to the importance of the region. With this, an increase in processing time is suppressed. In this operation, the importance is determined by using a facial recognition function and the region segmentation is performed.
An example of a region segmentation method that does not use the facial recognition function is described in “R. Achanta, et al., ‘SLIC Superpixels’, EPFL Technical Report no. 149300, June 2010”. In this document, the region segmentation is performed by setting the center point in each region resulting from the region segmentation and determining which center point of the surrounding region a target pixel is close to by using a pixel depth from the center point and a depth in a color space as parameters.
However, in Japanese Patent Application Laid-open No. 2010-117593, although an increase in processing amount is prevented by performing the region segmentation and changing the number of depth measurement points according to the importance of the region, in a case where the depth map of all pixels is needed, all pixels have to be measured so that the processing amount cannot be suppressed.
In addition, although the number of depth measurement points is changed according to the importance of the region, the shape of the region is rectangular, and hence there is a problem that depth measurement accuracy is reduced in a case where objects having different depths are included in the rectangular region.